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EXAMINER'S ANSWER 

This is in response to the appeal brief filed 12 June 2003. 

(1) Real Party in Interest 

A statement identifying the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

A statement identifying the related appeals and interferences which will directly affect or be 
directly affected by or have a bearing on the decision in the pending appeal is contained in the brief. 

(3) Status of Claims 

The statement of the status of the claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

( 5) Summary of Invention 

The summary of invention contained in the brief is correct. 

(6) Issues 

The appellant's statement of the issues in the brief is correct. 

(7) Grouping of Claims 

Appellant's brief includes a statement that the rejection of claims 1-63 stand 
or fall together. 
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(8) Claims Appealed 

The copy of the appealed claims contained in the Appendix to the brief is 

correct. 

(9) Prior Art of Record 

The following is a listing of the prior art of record relied upon in the rejection 
of claims under appeal. 

6,166,738 ROBERTSON et al 12-2000 

5,669,006 JOSKOWICZ et al. 9-1997 

5,701,444 BALDWIN 12-1997 
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(10) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 
Claim Rejections - 35 U.S.C. § 103 

I. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious at 
the time the invention was made to a person having ordinary skill in the art to which said subject 
matter pertains. Patentability shall not be negatived by the manner in which the invention was 
made. 

II. Claims 1-3, 5-9, 11, 12, 14, 16-22, 24, 33-40, 42, 44, 46-51, 53, 54, 56, and 
58-63 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Robertson et al (6,166,738) and Joskowicz et al (5,669,006). 

The examiner has carefully considered all the claims 1-63. The present 
invention relates to a user interface to objects. The present invention exploits the 
spatial memory of properties of an object. 
Per claims 1 and 47: 

As can be seen from the cited reference, the Prior art is basically identical to 
the present invention in that the Robertson patent discloses a graphical user interface 
in which object thumbnails are rendered on a simulated three-dimensional surface as 
in the present invention. Robertson discloses a man-machine interface method for 
permitting a user to act on thumbnails (see claim 1 of Robertson's); 

as per "generating a three-dimensional environment (see claim 16 of 
Robertson's)"; 
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as per "determining a two-dimensional location (see claim 1 of 
Robertson's)"; 

as per "generating the thumbnails within the three-dimensional 
environment" (see claim 1 of Robertson's). 

While Robertson patent discloses a perspective view facility 252 to 
determine the thumbnail object location or position information (col 17, lines 29-61), 
the location or position information of Robertson's does not explicitly show further 
information about the object, such as the depth parameter of the object thumbnail on 
the three dimensional surface is not explicitly shown. 

However, Joskowicz et al (5,669,006) discloses this shortcomings. The 
Joskowicz patent relates to generating a spatial layout of the visible segments on a 
computer display screen. Joskowicz further discloses obtaining a screen layout for a 
given set of object ("clique") is to find the locations of its associated episodes and 
depths (i.e., Z-ordering) on the screen (col 4, lines 3-34, 52-67). Furthermore 
Joskowicz patent discloses the Z-ordering defines the relative depths for overlapping 
episodes as they appear on the display screen. The Joskowicz patent further 
formulate and defines the Z-ordering of episodes according to their relationship, or 
according to their arrangement or priority in the Z-ordering list. Thus, Joskowicz 
patent teaches relationship value (parameter) (col 4, lines 3-34) between episodes, 
wherein each episode is represented as a thumbnail (fig. 1). 

Therefore, it would have been obvious to one having ordinary skill in the art 
at the time of the invention was made to incorporate the three-dimensional features 
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of an object, such as depth information of Joskowicz's with the Robertson's three- 
dimensional environment because the incorporation will enable Robertson to display 
or edit the values of the thumbnail associated with the displayed object (such as the 
document shown in Figs 8A-18). Thus, the combination of the Robertson reference 
and Joskowicz reference would result in the invention recited in claims 1 and 47. 
Per claim 34: 

as per "a system which permits a user to interact with thumbnails,..." (see 
claim 1 or 23 of Robertson's); 

as per "an input facility for accepting user inputs" (see claim 23 of 
Robertson's); 

as per " a storage facility containing a two-dimensional location, cursor 
location a location information and state information ..." (see col 15, lines 39-64, 
claim 24 of Robertson's); as per "a processing unit which accepts user inputs, 
updates a two-dimensional location, cursor location, position or location 
information" (see claim 23 of Robertson's); 

as per "generating video output and a video display unit" (see claim 23 of 
Robertson's). 

As mentioned in claim 1 , While Robertson patent discloses a perspective 
view facility 252 to determine the thumbnail object location or position information 
(col 17, lines 29-61), the location or position information of Robertson's does not 
explicitly show information about the object, such as the depth parameter of the 
object thumbnails on the three dimensional surface is not explicitly shown. 
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However, Joskowicz et al (5,669,006) discloses this shortcomings. The 
Joskowicz patent relates to generating a spatial layout of the visible segments on a 
computer display screen. Joskowicz further discloses obtaining a screen layout for a 
given set of object ("clique") is to find the locations of its associated episodes and 
depths (i.e., Z-ordering) on the screen (col 4, lines 3-34, 52-67). 

Therefore, it would have been obvious to one having ordinary skill in the art 
at the time of the invention was made to incorporate the three-dimensional features 
of an object, such as depth information of Joskowicz's with the Robertson's three- 
dimensional environment because the incorporation will enable Robertson to display 
or edit the values of the thumbnail associated with the displayed object (such as the 
document shown in Figs 8A-18). Thus, the combination of the Robertson reference 
and Joskowicz reference would result in the invention recited in claim 34. 
Per claim 2: 

as per "... the depth is a linear function of at least one parameter of the object 
associated with the thumbnail" (see Joskowicz: col 4, lines 52-col 6, lines 62). 
Per claim 3: 

as per "... the depth is a polynomial function of at least one parameter of the 
object associated with the thumbnail" (see Joskowicz: col 4, lines 52-col 6, lines 62). 
Per claim 5: 

Robertson and Joskowicz disclose that object may be related or rendered on 
to user's explicit selection or based on a property , such as age, storage location, etc. 
(See Robertson: col 7, lines 12-30). 
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Per claims 6 and 48: 

As per " accepting inputs from the user input device; determining a two- 
dimensional cursor location haled oil the accepted inputs; and generating a cursor at 
the determined two-dimensional cursor location, to be rendered on the video display 
device." Robertson and Joskowicz disclose the limitations recited in claims 6 and 48 
(see claims 5 and 6 of Robertson's). 
Per claims 7 and 49: 

As per "if the two-dimensional location of the cursor is located on or over 
one of the thumbnails, defining a state of that thumbnail as active." Robertson and 
Joskowicz disclose the limitations recited in claims 7 and 49 (see col 23, lines 30-40 
of Robertson V). 
Per claims 8 and 50: 

As per "generating a pop-up information bar located over the active 
thumbnail, to be rendered on the video display device." Robertson and Joskowicz 
disclose the limitations recited in claims 8 and 50 (see col 17, lines 62-col 18, lines 5 
of Robertson's). 
Per claims 9 and 5 1 : 

As per "if the user input provides a selection input and if an active or 
floated thumbnail exists, then generating a higher resolution visual representation of 
the object represented by and associated with the active or floated thumbnail, at a 
preferred viewing location at a foreground of the three dimensional environment, to 
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be rendered on the video display device." Robertson and Joskowicz disclose the 
limitations recited in claims 9 and 51 (see claims 8 and 10 of Robertson *s). 
Per claims 11 and 53: 

As per "generating an animation which moves the higher resolution visual 
representation of the object represented by and associated with the active thumbnail 
from the location of the active thumbnail to the preferred viewing location at the 
foreground of the three dimensional environment, to be rendered on the video 
display device." Robertson and Joskowicz disclose the limitations recited in claims 
1 1 and 53 (see claim 10 of Robertson 's). 
Per claims 12 and 54: 

As per "if the user input provides a deselection input and if a selected 
thumbnail exists, then generating a video output for moving the high resolution 
visual representation of the object represented by and associated with the active 
thumbnail to the two-dimensional location of the selected thumbnail, to be rendered 
on the video display device." Robertson and Joskowicz disclose the limitations 
recited in claims 12 and 54 (see claim 11 of Robertson 's). 
Per claims 14 and 56: 

As per "if the user input provides a selection input and if an active 
thumbnail exists, then I) invoking an application related to the object represented by 
and associated with the active thumbnail, ii) loading the object represented by and 
associated with the active thumbnail into the application, and iii) generating a video 
output of the application with the loaded object represented by and associated with 
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the active thumbnail at a preferred viewing location, to be rendered on the video 
display device." Robertson and Joskowicz disclose the limitations recited in claims 
14 and 56 (see claim 13 of Robertson s). 
Per claim 16: 

As per " if-the user input provides a move input and if an active or floated 
thumbnail exists, then I) updating the two-dimensional location of the active or 
floated thumbnail based on the move input." Robertson and Joskowicz disclose the 
limitations recited in claim 16 (see claim 1 or 23 of Robertson 's). 
Per claim 17: 

As per "the move input is a left button mouse drag." Robertson and 
Joskowicz disclose the limitations recited in claim 17 (see claim 15 of Robertson f s). 
Per claims 18 and 58: 

As per "the three-dimensional environment defines a foreground and a 
background, and wherein the act of generating thumbnails, within the three- 
dimensional environment, at the determined two-dimensional location and depths, to 
be rendered on the video display, includes) using perspective views so that any 
thumbnails in the foreground defined by the three-dimensional environment appear 
larger than any thumbnails in the background defined by the three-dimensional 
surface." Robertson and Joskowicz disclose the limitations recited in claims 18 and 
58 (see col 17, lines 1- 61 , claim 16 of Robertson 's). 
Per claims 19 and 59: 



Application/ConflPNumber: 09/335,640 
Art Unit: 2173 



Page 1 1 



As per "a thumbnail partially occludes any thumbnails behind it, base on a 
viewing point." Robertson and Joskowicz disclose the limitations recited in claims 
19 and 59 (see claim 17 of Robertson 's). 
Per claims 20 and 60: 

As per "d) accepting inputs from the user input device; e) determining a 
viewing point two-dimensional location, depth and direction based on the accepted 
inputs; and generating only that portion of the three-dimensional environment and 
only those thumbnails that are in front of the virtual viewing determined in act e) 5 to 
be rendered on the video display device." Robertson and Joskowicz disclose the 
limitations recited in claims 20 and 60 (see claim 18 of Robertson 's). 
Per claim 21: 

As per "the thumbnails are low resolution bit maps." Robertson and 
Joskowicz disclose the limitations recited in claim 21 (see col 12, lines 36-53 of 
Robertson V). 
Per claim 22: 

As per "the low resolution bit maps are 64 pixels by 64 pixels and have 24 
bit color." Robertson and Joskowicz disclose the limitations recited in claim 22 ( see 
col 12, lines 36-53 of Robertson 's). 
Per claim 24: 

As per "for each of the thumbnails, determining a shade to be applied to the 
thumbnail based on its depth." Robertson and Joskowicz disclose the limitations 
recited in claim 24 ( see col 6, lines 39-55, col 9, lines 20-34 of Robertson 's). 
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Per claim 33: 

As per "the three dimensional environment includes a floor, the method 
further comprising a step of generating a shadow, for each of the thumbnails, on the 
floor." Robertson and Joskowicz disclose the limitations recited in claim 33 ( see col 
6, lines 39-55, col 9, lines 20-34, col 12, lines 57~col 13, lines 9 of Robertson 's). 
Per claim 35: 

As per "the state information for each the thumbnails contained in the 
storage facility includes an indication of whether or not the thumbnail is active." 
Robertson and Joskowicz disclose the limitations recited in claim 35 ( see claim 24 
of Robertson's). 
Per claim 36: 

As per "the storage facility further contains descriptive textual information 
for each of the thumbnails, and wherein, if a thumbnail is active, I)the processing 
unit generates a pop-up bar, based on descriptive textual information, for the active 
thumbnail, and ii) the video display unit renders the pop-up bar over the rendered 
thumbnail." Robertson and Joskowicz disclose the limitations recited in claim 36 ( 
see claim 25 of Robertson V). 
Per claim 37: 

As per "if a thumbnail is active or floated and the input facility accepts a 
selection input, then I) the processing unit updates the state of the thumbnail, ii) the 
processing unit gets a second, higher resolution, visual representation of the object 
represented by and associated with the thumbnail, iii) the processing unit generates a 
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video output based on the higher resolution, visual representation of the object 
represented by and associated with the thumbnail at a preferred viewing location, 
and iv) the v ideo display device renders the video output generated by the 
processing unit." Robertson and Joskowicz disclose the limitations recited in claim 
37 ( see claim 26 of Robertson 's). 
Per claim 38: 

As per " an audio output device, wherein the storage facility further 
contains a first audio cue, and wherein, when an object is selected, the processing 
unit provides the first audio cue to the audio output device." Robertson and 
Joskowicz disclose the limitations recited in claim 38 ( see claim 27 of Robertson 's). 
Per claim 39: 

As per "each thumbnail is a 64 pixel by 64 pixel bit map having 24 bit 
color and wherein each higher resolution, visual representation of the objects is a 512 
pixel by 512 pixel bit map having 24 bit color." Robertson and Joskowicz disclose 
the limitations recited in claim 39 ( see claim 28 of Robertson 's). 
Per claim 40: 

As per "the processing unit further effects a video output based on an 
animation of the higher resolution, visual representation of the object represented by 
and associated with the thumbnail, moving from the location of the thumbnail to a 
location at the foreground of the three-dimensional environment." Robertson and 
Joskowicz disclose the limitations recited in claim 40 ( see claim 29 of Robertson 's). 
Per claim 42: 
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As per "if a thumbnail is active and the input facility accepts a selection 
input, then I)the processing unit updates the state of the thumbnail to select selected, 
ii) the processing unit opens an application with which the object, associated with 
and represented by the selected thumbnail, is associated, iii)the processing unit 
generates a video output based on the object loaded onto the opened application and 
a preferred viewing location, and v) the video display device renders the video output 
generated by the processing unit." Robertson and Joskowicz disclose the limitations 
recited in claim 42 ( see claim 13 and 23 of Robertson 's). 
Per claim 44: 

As per "if a thumbnail is active or floated and the input facility accepts a 
move input, then I) the processing unit updates the state and location of the 
thumbnail, ii) the processing unit generates a video output based on the updated 
location of the thumbnail, and iii) the video display device renders the video output 
generated by the processing unit." Robertson and Joskowicz disclose the limitations 
recited in claim 44 ( see claim 23 of Robertson 's). 
Per claim 46: 

As per "the storage facility further contains virtual viewing point location 
information, wherein the input facility includes a mouse, and wherein the processing 
unit d) accepts input from the user input device; e) determining a viewing point 
location and direction based on the accepted inputs; and f) generates only that 
portion of the three-dimensional environment and only those thumbnails that are in 
front of the virtual viewing point determined in step e), to be rendered on the video 
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display device." Robertson and Joskowicz disclose the limitations recited in claim 46 
( see claims 18 and 35 of Robertson 's). 
Per claims 61 and 63: 

Claims 61 and 63 correspond generally to independent claim 1 and recite 
similar features in a method and in machine readable medium form respectively, and 
therefore are rejected under the same rationale. 
Per claim 62: 

Claim 62 corresponds generally to independent claim 34 and recites similar 
features in a system form, and therefore are rejected under the same rationale, 
f. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Robertson et al (6,166,738) and Joskowicz et al (5,669,006) and Baldwin 
(5,701,444). 

claim 4 recites a depth represented in exponential function of at least one 
parameter of the object associated with the thumbnail. While Robertson and 
Joskowicz discloses depth represented linear and quadratic functions, representing 
depth in exponential function not shown. 

However, '444 discloses depth represented in exponential function (col 45, 
lines 16-22). Therefore, it would have been obvious to one having ordinary skill in 
the art at the time of the invention was made to incorporate exponential function of 
'444 with Robertson and Joskowicz's depth representing functions (linearly, and 
quadratically), thus, this incorporation helps in providing a plurality of different 
depth functions applied with the Robertson and Joskowicz's system. 



/ 
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(11) Response to Arguments 

Applicant's arguments filed 12 June 2003 have been fully considered but they 
are not persuasive. 

In response to applicant's argument that there is no suggestion to combine the 
references, the examiner recognizes that obviousness can only be established by 
combining or modifying the teachings of the prior art to produce the claimed 
invention where there is some teaching, suggestion, or motivation to do so found 
either in the references themselves or in the knowledge generally available to one of 
ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 
1988)and In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, 
Robertson suggests that his invention may employ perspective views of an object 
thumbnail (perceived image scaling with distance) on a three-dimensional surface 
(col 6, lines 39-55) and further suggests object may be related based on a property 
(age, storage location, etc) (col 7, lines 12-30). Joskowicz also teaches depth 
associated with parameter such as priority or arrangement of an object (episode) 
associated with thumbnail (fig. 1). Thus, since both Robertson and Joskowicz are 
directed to the same field of endeavor, the combined art will enable user to provide a 
user interface with more enhances feature ( i.e., depth as a function of parameter), 
and the graphical user interface further enables user to easily manipulating, viewing 
and organizing objects in a display. 

Appellant further argues that "neither Robertson, et al nor Joskowicz, et al , 
alone or in combination, teach or suggest the claimed feature of applicant." Such 
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argument includes "determining a two-dimensional location and a depth of each of 
the thumbnails in the three-dimensional environment, wherein, for each of the 
thumbnails, the depth is function of at least one parameter of the object 
associated with the thumbnail" as recited in the Independent claim 1 as well as in 
claims 34, 61 and 62. In contrast to the appellant's argument the Robertson and 
Joskowicz teach the claimed subject matter. Specially, Joskowicz teaches a depth 
rendering process as a function of parameters; such function may be linear and such 
parameters include, among other things, arrangement of episodes represented by 
thumbnails, size of episodes represented by thumbnails, type ( video, audio, or text, 
etc) of episodes represented by thumbnails, priority or arrangement of episodes, and 
author interaction (col 4, lines 3-34, Fig. 9). Joskowicz further describes that 
obtaining a screen layout for a given set of object ("clique") is to find the locations of 
its associated episodes and depths (i.e., Z-ordering) on the screen (col 4, lines 3-34, 
52-67). Furthermore Joskowicz patent discloses the Z-ordering defines the relative 
depths for overlapping episodes as they appear on the display screen. The Joskowicz 
patent further formulate and defines the Z-ordering of episodes according to their 
relationship, or according to their arrangement or priority in the Z-ordering list. 
Thus, Joskowicz patent teaches relationship value (parameter) (col 4, lines 3-34) 
between episodes, wherein each episode is represented as a thumbnail (fig. 1). Thus, 
combining the (738) with ('006) results in determining the depth as a function of 
parameter of the object associated with the thumbnail 
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In response to applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant 
relies (i.e., "as values of properties or parameters of the objects change, the simulated 
depth at which they are rendered may change." "The simulated depth at which a 
thumbnail is rendered can change without user intervention"; "automatically adjusts 
the depth"; "as shown in Figure 3 an object record 304 may include an object 
identifier field 306 which include a value used to distinguish the object from all other 
objects.") are not recited in the rejected claim(s). Although the claims are interpreted 
in light of the specification, limitations from the specification are not read into the 
claims. See In re Van Geuns, 988 F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

Appellant states that Joskowicz et al teaches rendering an icon as a function 
of a parameter associated directly with the icon, i.e., the depth of the icon is directly 
related to the physical size of the icon (height times width)." On the other hand, 
Appellant, repeatedly argues that Joskowicz, et al simply teaches rendering of icons 
(including Z-axis based rendering) as a function of the icons themselves (e.g., size, 
area, predefined constraints relating to spacing of the icon, but not as a function of at 
least one parameter of an object associated with the displayed thumbnail as defined 
in the claimed invention." In contrast to Appellant argument Robertson and 
Joskowicz teach the depth as a function of parameter (such as priority or 
arrangement of episodes) of the object associated with the thumbnail. 

Appellant also argues about the rejection of claim 4, and states that since 
claim 4 depends on claim 1, Baldwin fails to make up for the aforementioned 
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deficiencies of Robertson and Joskowicz, with respect to claim 1. In contrast to the 
appellant argument, the rejection still stands because Robertson and Joskowicz teach 
the claimed invention as described above in claim 1. 

For the above reasons, it is believed that the rejections should be sustained. 



t 
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Respectfully submitted, 
7«te 3<«<& 22 August 2003 
USPTO, Washington, DC 
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